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/iBSTRACT 


Artificial intelligence techniques have been applied 
to the design of control systems. In the present work a 
synthesis technique known as off-line (learning) training is 
considered, which achieves a practical realization of closed 
loop time optimal, fuel optimal, or minimum energy control 
laws. In this technique a training set in state space is 
obtained. A training set is a set of quantum regions of the 
state space for which the optimal control is known. A code 
is assigned to all such possible regions in order to identify ' 
a particular zone. The training set is used to train a 
learning controller which synthesizes the complete control 
law during the training by means of a learning algorithm. 

The function of a trained controller is to provide the optimal 
control for all points in state space with a limited training 
set. 

In the present work the experimental results of the 
performance of the learning controller using different codes 
are obtained. A learning algorithm called the "relaxation 
algorithm" is also considered in order to investigate the 
learning characteristios of a learning controller. Experi- 
mental results are given for the -same. 
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The applioation of a trainable controller is 
finally considered in a third order tiine optimal control 
, system. The training set is obtained by generating optimal 
trajectories in the state space using the Neustadt's method 
and the quantum zones are identified for the training set 
through which the optimal trajectories pass. The experi- 
mental results are provided, demonstrating the performance 
and reliability of a trained controller using different 
codes. 

All computer programs were written in FORTRAN IV 
language to be run on IBM 7044 computer. 
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CHAPTER 1 

THE OPTIMAL CONTROL PROBLEM 

1 . 1 Introduction; 

All physical processes described by a cause— effect 
relationship can be viewed as a system with inputs and out- 
puts. 

Let us assume that we are given a system, or, . a 
"black box”, to which we can apply certain input signals, in 
order to observe and measure the resultant output signals. 

Our ultimate objective is the determination of an input which 
will produce an output with certain desired characteristics, 
and which will in doing so, minimize the "cost" of operation 
(the term cost could be the , time required fuel consumed or 
energy consumed or other indices of interest). We might try 
to achieve our objective by a trial and error procedure, by 
trying first one input and then another. But in general, 
this procedure will not guarantee a solution, besides being 
time consuming. However, we might look upon our efforts as 
experiments which could lead us to a description of the 
behaviour of the system and could indicate to us the i«.rti- 
cular output resulting from the application of a particular 
input. We generally attempt to solve these problems by 
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^e-veloping mathematical models in which equations relate the 
output to the input. 

We shall consider systems which can he described 
by ordinary differential equations. We call such systems as 
Dynamical Systems, These can be represented mathematically 
as 


*1 = ••• V "1> "2 ••• V 

9 


Cn = ^2 ^ n * ^^2 *'* "^ r ^ 


r (1.1) 


'There x-j , Xg ... x^ are the n state variables of the system 
and , U2 ... are r input variables. The dot notation 
represents the time derivatire. 

If the system of above differential equation is 
represented in vector form then we have - 


X(t) « I.(X(t), U(t), t) 


( 1 . 2 ) 


If the dynamical system under consideration is 
linear, then (1,2) takes the form as - 


X(t) s= A(t) ^(t) + B(t) U(t) 


< 1 . 3 ) 
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*1 • 2 The Optimal Control. Problem; 

In all control problems We are interested in 
transferring the state of the system to a particular state 
from some initial state by application of proper control 
inputs. If the system is controllable then this can be 
achieved by applying suitable control Inputs, 

In order to evaluate the perfonnance of a dynamical 
system quantitatively, we select a measure of perforii©.nce. 

An optimal control is defined as one that minimizes (or 
maximizes) the performance measure. The selection of perfor- 
mance measure depends on the nature of problem at hand* For 
example, if the problem is to "move” the system from an 
initial state XCt^) to a final state X(tj^) in minimum time, 
this is known as a time optimal control problem, where the 
control input has to be so chosen so as to minimize the 
performance measure which is the integral of time or - 



J being the performance n^asure. 

Similarly, in some other problem wO may be required 
to "move" the system from an initial state 3C(t„) to a final 
state X(t^) such that the fuel cor^uraptioii is minimum, or 
in other words, if the control input |i(t) represents a 
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variable proportional to the rate of fuel consumption, then 
the problem is to minimize J, where - 

^ "i'*) (1.5) 

to 1^1 

oC being a constant of proportionality. 

The general linear optimal control problem is to 
find an admissible control U (t) which causes the dynamical 
system 

i(t) = A(t) X(t) + B(t) IJ(t) 

to follow an admissible f ajectoiy (a trajectory satisfying 
some constraints on state variables.) that minimizes the 
performance measure 

t^ 

J = hCX(tj), tj) + J g(X(t), 2(t), t) dt (1.6) 

u 

0 

The equation (1.6) is a general form of the perfor- 
mance measure and X(t^) and t^ are the terminal state vector 
and the terminal time. In the present work we shall consider 
the performance measures dealing with the minimum time, fuel 
and energy control problems. 

Let us consider the example of a spacecraf t enterin." 
the earths atmosphere. During the atmospheric re-entry, 
there exists an optirniim entry angle that guarantees a safe 



5 


landing. If this re-entiy angle is too large, the spacer 
craft inay be bounced by the outer layers of atmosphere back 
to outer space, and if it is too low, then the spacecraft 
may get over-heated and burnt due to air friction. Another 
effect noted is over-heating of the spacecraft body due to 
high velocity attained with incorrect re-entry angle. The 
gas in the boundary layer between the shock-wave front and 
the spacecraft surface becomes ionised. A high temperature 
plasma is thus formed around the space vehicle, which may 
fully envelope it and the high frequency radio communication 
units It carries. This envelope of plasm can completely 
out off the radio communication of the spacecraft with the 

ground station and thus re-entry can not be program controlled 
by ground station. 

If during re-entry, the entry angle is disturbed 
beyond the safe limits, it is essential that this error be 
corrected in minimum possible time considering the great 
speed of spacecraft descent. Here the problem is to choose 

the proper control inputs to correct the error in minimum 
time, 

:S Another exan^le is a fuel optimal control system. 

In case of moon landing modules, the problem of minimum 
expenditure of fuel is very important, let us assume that 
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a spacecraft is in the terminal descent phase of a lunar 
mission, and is descending vertically. The problem is to 
properly switch on or switch off the retro-motors in such a 
way that the module attains zero terminal velocity at some 
specified altitude with a miiiiiiiuiQ expenditure of fuel. Once 
the terminal condition is achieved the vehicle can hover 
by applying a thrust acceleration of one lunar g (acceler- 
ation due to gravity). Such a mission permits the inspection 
of the moon's surface and a suitable landing place can be 
selected. 

In practical situations, such as the spacecraft 
problem, there are some constraints on the control inputs, 
for example the maximum thrust offered by the motors is 
always limited. In such cases we have the constrained input 
optimal control problem. 

1 , 3 Theoretical Methods in Optimal Control; 

jB'rom an historical point of view, the need for 
theory in the area of optimization arose, to a large degree, 
from the stringent requirements of aerospace systems. Such 
systems are inherently nonlinear and they are subject to 
constraints, i.e, limited thrust capability, Bcceleration 
constraints, fuel limitations, limited fin steering avail- 
ability etc. Before the development of modem control 



7 


theory, the only body of theory available for control was 
that dealing with the analysis and design of servomechanisms. 
However, the theory of servomechanisms was not adequate for 
the analysis and design of unconstrained single-input-output, 
linear and time invailant systems. Moreover, most of these 
methods were focused upon the relative stability of the 
closed loop systems, the design tools (e.g. root locus, 
frequency response, describing fiuiction etc.) were graphical 
and qualitative in nature, and the design process was 
mostly done in a trial and error fashion, under the assumption 
of zero initial conditions. The Euler’s equation method 
using the calculus of variation was applied to control 
problems, but it did not prove very useftil for problems with 
constrained inputs. 

The two D^in theoretical approaches to the control 
optimization problem have been (i) Bellman’s Dynamic 
Programming method £ 1 ^ which is based on the principle of 
optimality and (ii) Pontryagin’s Minimum Principle £ 2 ^ which 
can be viewed as an extension and application of classical 
calculus of variation to the optimal control problem. 

The Dynamic Prograianing method was originally 
developed for discrete time problems and later on it was 
applied to continuous time problen© . In the later form it 
is often referred to as the Hamilton-Jacobi-Bellman theory. 
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Its major disadvantage lies in the large computer memory- 
requirements ("The curse of dimensionali-ty" ) • 

The Minimum Principle was originally developed for 
continuous time prohlems, and has been extended to discrete- 
time problems. Its major disadvantage is that it provides, 
in general, only local necessary conditions for optimality. 
Its computational requirements are not as severe as those 
associated with dynamic programming. 

The minimum principle, however, proves to be a very 
powerful method in investigating the na-tuie of the optimal 
controls. 

1 ,4 Application of Pontryagin's Minimum Principle to 

Investigalie' the Ifature of Controls for Minimum Time, 

Fuel and Energy Problems : 

As we have stated earlier, the minimum principle is 
a very powerful method to obtain a qualitative idea about the 
nature of optimal control, we- will illustrate its use in 
time, fuel and energy optimal control problems. A detailed 
discussion of the minimum principle can be found in 
Pontryagin|* 2^, or Athans arjd Palb [4]. 

1.4.1 Time-optimal-control problem; 

Consider a linear time invariant system 

i(l3) - 


A X(t) + B U(t) 


(1.7) 



where JC is an n-vector, and U an r-veotor. A and B are 
constant matrices. The constraint on U is 

|Ui(t) I < 1 , 1 = 1,2, ...,r (l.g) 


The solution of ( 1 . 7 ) is given by 
X(t) = e" . r(0) + e^'* / u(z) a c 


( 1 . 9 ) 


If the problem is to drive the system to zero state 
in time t^, then we have 

- ■‘^f ■ 

^ -At 


( 1 . 10 ) 


= - 1 " B U(^) d r 

* 0 

A control which satisfies (1.10), driwes the system 
from its initial state X( 0 ) to 0. 

The adjoint eqyation of the system (I.7) is given by, 

T(t) = -A^^(t) 

where 0 (t) is an n-veotor and A* is the t2?anspose of A. The 
solution of (1.11) is given by 

O (t) = (0) 

where (0) is a constant. 

The performance measure is given by 

J = I 1 . d* 
o' 


( 1 . 12 ) 


let us define Hamiiltonian as 
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If an optimal control U*(t) exists, then according 
to minimum principle, U*(t) must minimize the Hamiltonian 
for all t, 0 ^ t ^ tjp 2,3 • Thus 

rI(U*(t),v^*^(t), X*(t))< H(U(t),v^*^(t), X*(t)) (1.14) 

or, 

y ^^(t) [a X*(t) + B U*(t)] < v^*^(t) [a Xf (t) + B H*^(t)J 

“ (1.15) 

Thus from the equation (1,15) we notice that if no 
element ofij|/* (t).B vanishes for all t, 0< t < t^^; then the 
control that satisfies the equation (1.15) is given by 

u^(t) = - Sgn [^] » i = 1, 2, ... r (1.16) 

where q^^ are the elements of the vector 

a = »''b 

Brom the equatioh (1,16) the bistable nature of the optimal 
control is clear .from Big. 1.1, 

1,4.2 Bue 1-optimal-control problem: 

The fuel optimal control problem has the following 
four forms, 

(i) The fuel consiin^tion is minimized without 
any restriction on the response time, the response time being 
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the time required to drive the system from any initial state 
X(0) to a final state X(t^). 

(ii) The fuel consumption is minimized with the 
response time fixed a priori. 

(iii) The fuel consumption is minimized to drive 
the system from any initial state X(0) to a final state 
X(t^) such that the response time t^ is a multiple of the 
minimum time t*; t* being the minimum time to drive the 
system from the same initial state to the same final state, 
without any constiraint on the consumed fuel. 

(iv) A linear combination of fuel and the response 
time is minimized. 

We will restrict ourselves to problems where 
response time is not fixed a priori. 

Consider the system (1,7) 

X(t) = A X(t) + B U(t) (1.7) 

The performance measure to be minimized has the 

form, 

f^f r 

J = J H dt (1.17) 

o i=l 

Hotice that u^(t) is a measure of the rate of 


fuel consul^) tion 



15 


The Harniltonian is defined as, 

1C 

H = i + 2 }Uj^(-fc) 1 (1.1£ 

~ i='1 

Substituting the expression f or X in (1 .t8) , we 

have 


3? 

H = ^[a X + B U j-i- 2^ |Uj,(t) I (1.19; 

Let 

S. ~ ® Q.>j j 9.2 > ••• 9^ (l»20j 


Mow according to the mininium principle, if there 
exists an optiinal control U*, then it must minimize the 
Hamiltonian for all t* 0 t £ t^* In other words, U* should 
minimize the expression, 

B = 9j_ + I I » i = 1 , 2, . r (1.21) 


We have. 


E = 


[l + if is positi^ 


(1 . 22 ) 1 


[ |-1 + ^ negative 


notice from (1.22) that if ; >1 , then minimum 

value of E is obtained if u^ = - Sgn . Similarly, if 
q . < -1 then minimum value of B is obtained if u . = 

3- ^ X ' ■ ■ 

- Sgii[qi] . 



u 


Uow if -1 <, +1 , then we find out that E is 


minimized if u^ = 0. 


Thus the optimal control is given by, 


Uj_ = - Sgn q^: , I q;i| > 1 


n, = 0 

X 


, if -'* < < 

for i = 1 , 2, * . . r 


+1 


(1.23! 


Thus the tristable nature of the optimal control 
is clear from Pig, 1.2, 

1 . 4.3 Minimum-energy-oontrol problem: ! 

A class of problems exists where the square of the ^ 

I 

control signal is proportional to power, and its time integral I 

i 

is a measure of total energy used, ! 

I 

Consider the state equation (1.7) 1 


X(t) = A X(t) > B U(t) 


(1.7) 


We have the performance measure 


= 


'f r 


u?(t) dt 


o i=i1 

We define the Hamiltonian as 


(1.24) 


H 


(1.25) 
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or, 

H ^ [ A X + B 2] + i X (1.26 

We notice from equation (1,26) that the expression 
® ~ i Ui(t) + q^ u^(t) 

where q^ is as defined in (1.20) will be minimized if u^(t) 
is so chosen that, 

u^(t) = - 1 , if q^>1 

Uj^(t) = + 1 , if ^ -1 

and, u^(t) = - q^ , if -1 ^ q^ <+1 

Thus from Pig. 1.3, the nature of energy optimal 
control is clear. 



Implementing the Results of the Optimal Control Theo: 


In the previous section we have shown that the 
optimal control vector U*(t) depends on th4 solutiop. of the j 
adjoint equation Of the original state equktion. The solution 

4 jr*(t), of the adjoint equation depends on an initial value, .ir. i 

1 ■ ■ , 

V|j’. ^ (0) and this initial Value, depentis in 6ome unknowh manner on 
the initial state vector X(0), We Should point out that Upto 




Fig, 1.3 Energy-Optimal-Control 



Fig. 1,4 Implement ing the Optiml Control laws 
by On-Line Computer 
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date there are no formulae available to relate!s|^‘'0), to 
X(0). However, there exist a number of iterative methods to 
coiriputeH|/(0) and the optimal controls. Pleint 5~j gives a 
^ood account of these methods. These methods usually start 
with an initial guess ofS|^(0) and by successive approximation 
finalt|^ ( 0 ) is obtained. The method due to Neustadt 3 ^ 
appears to be particularly attractive as it can be readily 
programmed on a digital computer. 

One method that one can suggest to implement the 
optimal control laws, is to use an on line digital computer 
which can calculate^ (O) by using some iterative algoiithms C5^ 
and generate the optimal control in real time. It will be 
noticed that such a method tends to be of an open loop type 
as time happens to be an independent variable. We know that a 
more reliable system will be obtained if the optimal control 
is generated as a function of the state variables, as it 
will constitute a closed loop system, Athans 4 ^ points out 
that generally for systems of order higher than 3, the 
optimal control theory yields optimal controls as a function 
of time, and it is very difficult to find out a closed loop 
solution for controls as a function of state variables. 

As mentioned earlier to implem^t the optimal^^^^^^^^^^ 
control laws, an on line computing scheme can be used, but 
such a scheme may not be. possible due to the following 


reasons 
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(i) In applications such as re-entry of space- 
crafts, time can not be lost in mere computation, as time in 
such a case is very precious, 

(ii) Due to limitation of space, it may not be 
possible to install an on line computer. 

(iii) A reliable operation may not be guaranteed 
as optimal control generated will be function of time and 
not of state variables. 

A general block diagram of this method is shown in 

Dig, 1 .4. 

Another simple method to implement the optimal 
control laws is to use analog computers to generate optimal 
controls f43»{6^. This method is useful only when the order 
of plant is small and optimal control is known as a function 
of state variables, with simple switchii^ surfaces. We shall 
demonstrate this method for a simple problem of second order, 
and single input. 

Let the system dynamics be 
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Let the perfo.rmance index be 



Using the- mdUiiiBum pzinoiple -and soroe manipulations 
have the closed form solution for the switching trajectory as, 




- 2 Xg X 2 


This equation describes a partitioning curve in the 
state space, with the states of same optimal control on one 
side and, the states of the opposite optimal control on the 
other. The optimal control is bang-.bang type. I'igure 1.5 
depicts the switching curve with zones of constant control. 

The above optimal control law can be implemented using the 
analog computer elements as shown in the Pig. 1,6, 

1 -6 Cone lus ions t 

It is clear from the above discussion that the 
present implementation schemes are inadequate in applications 
such as re-entry control of spacecrafts. It should be noted 
that the spacecraft re-entry physics is a subject, about which 
little is known to us. The outer layers of atmosphere have 
a random nature. In cases of re-entry in the other planets 
such as Mars, we know little about the dynamics of its 
atmospheric envelope. Thus we feel that in such cases 




Pig. I'* 6 Hardware implementation of 
time-optimal control law 


CQj-i. 
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there is a need of developing a control system technology to 

imtch these needs. The techniques of Artificial Intelligence 

find extensive use in control systems. In the present work 

the analysis and simulation of learning controllers is 
carried out. 
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CHAPTER 2 

ARTIE ICIAl IHTEIJ.IQEUCE AITO lEAHHIKC MAGHIHES 

2 . 1 Limitations of Optimal Control law Implementation: 

In Chapter 1 we mentioned the existing methods in 

optimal control theory, and in particular applied Pontryagin’s 

minimum principle to investigate the nature of optimal controls 

for time, fuel and energy optimal problems. We also pointed 

out that the minimum principle yielded only a qualitative 

idea about the controls involved, while it did not explicitly 

yield the optimal controls as a function of the present state 

variables. The optimal controls obtained by using the miniTOiiTn 

principle are generally- a function of time for high order S 

systems. It is easier to implement the optimal control laws 1 

if we know the control as a function of state variables, as i 

■' :'i 

the states may be readily measured. If the control as a | 

' i 

function of state variables is not known, then it is still [ 

. ! 

possible to implement the optimal control laws with control 

li 

i ■ 

as a function of time, but such a system tends to be oiien I 

■ 

loop and is unreliable. In the sequel, we would like to * 

point out that generally for higher dimensional systems, j 
a closed lo6p optimal control law is very difficult to 
obtain. 
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We have also noticed that the optimal control 
depends on the initial adjoint vector'^ ( 0 ), which in turn 
depends in some unknown manner on the Initial state vector 
X(0). Unfortunately no formulae yet exist to relate^^CO) to 
X(0), The procedures that are available are iterative in 

nature. It should be pointed out ttiat realization of optimal 

■# 

controls by using an on-line computer to calculatev|j»(0) for 
every X(0) and then generating the optimal control in real 
time may not be feasible in critical applications, such as 
the re-entry vehicle problem. Moreover, the processing 

computer system would be very complex. If the initial guess 

.-it 

otY(O) is not wisely chosen then it may take considerable 
— 

time for the computer to obtain the final value of ^ (0). 
Moreover, in order to calculate (0) we must know the exact 
system dynamics. In practice the systems involved are non- 
linear and may also sometimes defy proper characterization, 

2.2 Learning Machines and Artificial Intelligence ; 

In order to overcome the difficulties described 
above we can suggest several alternatives, For example, we 
can collect extensive data about the optimal controls for 
state vectors distributed Uniformly in the state space of 
interest and store it in a memory device. Such a collection 
of data can be done by actual computation on a computer 
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or, by ground simulation if the system dynamics is not very 
well-known. Application of such a method would not be 
economical as it would require a large memory storage. 

Besides there may arise difficulties such as the state 
variables in question not belonging to the data set stored 
in the memory. 

An alternative approach is to use an "Intelligent 
Oontroller" . By "Intelligent" we mean one that has the 
capability to "learn" a solution to the optimal control 
problem, and is capable of giving a favourable response by 
associating the event in question with events that are 
familiar. These so-called "Intelligent Controllers" are 
expected to have properties similar to hun^n learning. 

As far as the latter is concerned we can state 
that based on human experience the skill to perform a job 
depends in general on the training and experience, and it 
is because of man’s adaptive natiore that he is able to learn. 
In this context one may also ask the question! "Why are 
persons with more "experience" preferred over others?" It 
is because experienced people can take decisions more quickly 
and wisely than those who have lese experience. The exper- 
ienced persons have been found to have creative abilities 
and ability to handle situations, unseen in past. 
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Again, if we consider some of the classic exiperi- 
ments carried out hy Pavlov 7 Jon animals, it was noticed 
that the salivation which occurs in respoiase tS the view 
or smell of food, can also he elicited by a diffe3?ent 
stimulus such as by ringing a bell, if the ring;ipg precedes 
or accompanies the presentation of food for a number of 
trials. He called these responses to the substitute stimuli, 
" Conditioned Reflexes ”, in the present use the term applied 
is conditioned response, Pavlov wrote in 1927 [Tj: 

if the intake of food by the animal takes 
place simultaneously with the action of a neutral stimulus 
which has been hitherto in no way related to the food, the 
neutral stimulus readily acquires the property of eliciting 
the same reaction in the animal as would the sight of food 
itself. This was the case with the dog employed in our 
experiment with metronome , . , . 

..... after several repetitions of the property of 
stimulating salivary secretion and of evoking the motor 
reactions chamcteris tic of elementary- reflex 

We note that the critical relationship in the 
experiment was the time relation between the original stimulus 
(unconditioned) and the neutral (conditioned ) stimulus. 

The two had to occur close together in time with conditioned 
stimulus occurring ahead of unconditioned stimulus . We find 
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here that the dog had adapted itself to associate the ringing 
of bell with the service of food. We also note that such a 
behaviour was obtained with repetitive trials. Each time 
the trial took place, the unconditional stimulus was getting 
reinforced. 

Learning has been defined by psychologists as: the 
process by which behaviour is originated or changed through 
practice or training. 

Thus given a goal and a situation, ah animal can be 
conditioned to some response. It has been observed that 
reinforcement by reward or punishment are important factors 
in hastening the learning process. 

Similarly our objective is to construct machines 
which can change their internal structure in such a way that 
its response to a certain situation would be correct, had its 
previous response to the same situation been incorrect. It 
is desired that the response of the machine improves over 
successive trials. 

The function of an intelligent controller is: 

(i) to interpret the present event (the present 
state variables) by generating a response (control signal) 
by using its past experience. 

(ii) if the present response is correct then 
either reinforce the attributes to the response, or j'ust 
do nothing. 
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(iii) if the present response is incorrect then 
change the internal structure in such a way that on the next 
occurrence of the same event the response is correct. 

An important question can now be asked, ”Do such 
intelligent machines exist?” and if yes, "How can we construct 
them?” We may very well start with answering the question, 
"What is intelligence?” The dictionary ^ 8 j definitions are 

(i) capacity for understanding; and other forms 
of adaptive behaviour and 

(ii) knowledge of an event; circumstance etc.; 
received or imparted; news; information. 

Discussions of the definition of intelligence and 
artificial intelligence have been in progress ever since the 
ability of the digital computer to perform logical operations 
was compared with the living brain* 

If we consider intelligence to be a uniquely human 
attribute then by definition, this attribute can not be 
associated with a computer or any other artificial system. 

On the other hand if one argues that there is some 
fundamental physical limitation on the ability to compute, 
process, decide, and think which applies to the creation of 
machine intelligence, then these same limitations must also 
apply to human intelligence. That is, any theoretical 
limitations based on imiversal natural laws must be equally 
applicable to man and machine. 
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A more scientific approach to this question was 
proposed hy Turing 9 'j in 1950, Turing suggested that the 
question whether the digital machines are as intelligent as 
the human brain can be answered by experiment and observation 
in which the relative behaviour of a system (which by defini- 
tion, thinks and exhibits intelligent behaviour — namely 
human behaviour)is observed. Turing proposed a question- 
answer scheme in which a person could ask questions and receive 
answers via some remote terminal such as a teletype and the 
questions could be answered either by a digital machine or 
another man at the other terminal. If one could determine 
whether a man or a machine is answering the questions then 
one may question whether machines are as intelligent as man. 

On the other hand if one could not determine whether a man 
or a machine is answering the questions, then one may say 
that machines are as intelligent as man. This imitation game 
is one approach which has not yet been fully explored. 

Another common argument against machines being 
intelligent is that a computer can do only what it is told 
to do by its human programmer. But if we really examine the 
nature of human intelligence, then we are forced to admit 
that due to the adaptive nature of human brain or the 
capability to learn , the inherent tendency to survive . and 
his experience about nature, give him the of thought 
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and intelligence* The basic, built in characteristics of man 
coupled with the changes in him through experience generate 
the proper " program ” for successful performance in his 
environment, this "program" being subject to changes. The 
present intelligence in man may be through some evolution 
process, obtained by the experience of ages. 

Simon has suggested: "If a computer thinks, 
learns, and creates it will be by the virtue of a program 
that endows it with these capacities. Clearly this will not 
be a program — any more than the human is — that calls for 
highly stereotyped and repetitive behaviour independent of 
the stimuli coming from the environment and the task to be 
completed. It will be a program that makes the system's 
behaviour highly conditioned on the task environment — on the 
task goals and on the clues extracted from the environment 
that indicates whether progress is being made toward these 
goals. It will be a program that analyzes, by some means, 
itsown performance, diagnoses its failures, and makes changes 
that enhance its future effectiveness" . 

2.3 Foundations of Learning Ife-ohines : 

Shannon had demonstiated that abstrect 

logical operations such as addition, multiplication, 
negation, implication and equivalence could be implemented 


30 


physically by using electrical switching circuits. The basis 
of the suggested implementation is the fact that to each 
combination of abstract logical propositions linked by con- 
nectiveness like ’’Not", "And", or "Or"; there corresponds a 
circuit formed by connecting basic coirqponents such as the 
inversion (or negation), addition and multiplication switches. 
Shannon's work was the foundation of the modern digital 
computer theory. 

later during 1943, McCulloch and Pitts [15J, who 
were studying neurons* and neural netwoiks, became interested 
in using threshold logic elements like biological neurons in 
place of ordinary switches in logical networks. They were 
interested in finding out whether threshold logic elements 
can be used to realize various switching functions. Above 
all, they believed that since the living brain also works on 
the principles of logic (all-or-none principle), hence it 
should be possible to explain the intelligent behaviour of the 
brain by studying simpler networks with neurons. They found 
out that such networks with threshold logic elements could 
be used wiiih ease in realizing the logical operations. A 
very remaikable fact was that with neural networks having 

* Neurons are neirve cells which are functional units of 

brain. The cell body has several inputs called "dcedrites" 
and the output is a thin long fibre called "axon", Al^'e-tailed 
discussion of neurons can be found in Bccles [11] . 
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adjustable or alterable synapse* (so as to lower or raise the 
threshold value), the same network could realize an entirely 
new logical switching function with a new threshold value. 

This was the foundation of a new kind of machine which 
exhibited adaptive behaviour . The networks of neurons with 
adjustable synapse acquires a means of changing the mode of 
their internal organization to yield the prescribed behaviour 
even under widely varying conditions. As a result, these can 
be made to mimic Homeostasis (self adjustment in changing 
environment , M). Hebb suggested the following 

idea to explain adaptation and learning in the living brain, 
"when an axon of cell A is neaf enough to excite a cell B and 
repeatedly or persistently takOS part in firing it, some 
growth process or metabolic change takes place in one or both 
cells such that A’s efficiency as one of the cells firing B 
is increased". Consequently, permanent changes in the 
threshold excitation of the neuron can possibly occur, 
although neurophysiologists have not yet been able to demons- 
trate them in the biain. 

Thus we find that the idea of adjustable synapse 
proves to be useful one. Using this principle we can construct 


* Sympse is the point where an axon frcan a neuron ends on a 
dendrite of some other neuron. 
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networks of neuron-like structures to achieve adaptive 
behaviour. In the next section we shall describe some basic 
concepts of pattern recognition which are necessary for 

understanding the principles of learning control systems. 

2^,4 fhe Pattern Recognition Problem: Introduction ; 

Mathematically, pattern recognition is a classifi- 
cation problem. Consider for example the recognition of 
characters. We wish to design a system such that a hand- 
written symbol will be recognized as an "A", a "B” etc. In 
other words we wish to design a machine that will classify 
the observed character into one of 26 classes. The hand- 
written characters ' are often ambiguous, and there will be 
misclassifications of the characters. The major goal in 
designing a pattern recognition machine is‘"to have a low 
probability of error. 

There a 3 ?e many problems that can be formulated as 
pattern recognition problems. Bor example, in weather 
prediction, the weather nay be divided into three classes; 
fair, rain and possible rain, and one may like to predict 
the weather at a future date- into one of these three classes. 
In the recognition of electrocardiograms, the classes are 
disease categories and the class of normal subjects. 
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The input to a pattern recognition machine is a 
set of measurements, and output is the classification. fPor 
example in weather prediction, possible measurements could 
be atmospheric pressure humidity, temperature etc. 

In pattern recognition tenninology these measurement 
variables are called ** l'eatures ” . Generally one might expect 
to include all the possible features as input variables for 
a pattern recognition machine, but it may be "costly" to j 

include all features. It has been shown by Hughes [16] 
that as the number of features is increased beyond certain 
limit, the performance of the pattern recognition machine 
deteriorates. Moreover, some of the features may not be 
relevant at all, or some features may be dependent on some ] 

' I 

other features. For example, in the case of weather prediction S 
the proposed features could be temperature, pressure, month { 
of the year, humidity, the wind velocity or the condition of 
sky. But the relevant features could be temperature, I 

pressure, month of the year or wind velocity, as we know 
humidity depends on temperature and thus it is irrelevant. 

Apart from feature selection it may be necessary 
to process the input variables, to provide a convenient y 

input format. By processing, the problem of pattern recog- 
nition becomes easy. 
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In pattern recognition, we assume that each set of 

data to be classified ^ ^ 4 > 

^iiiea IS a set of n real numbers. S',, s 

Tn_ .... . ' ' 1 ' 2 » 


• s 


n* 


We call such a set a nattem A i • 

£_ — 1® — n. A device which sorts 


patterns into categories is called a pattern 

A general block diagram of a pattern recognition 
system is shown in Fig. 2.1. 


To illustrate further some more ideas of pattern 

recognition consider a lumber mill prcduclng assorted hardwood 

planks that wants to automat the process of sorting finished 

lumber according to the kind of tree. If it is required to 

sort teakwood from rosewood, we may check whether the wood 

sample is hard or soft, or whether it is coarse grain or fine 

grain or even by checking its weight. All these properties 

are the features of a wood sample. One nethod is shown in 
Fig. 2.2. 

If we take a number of pieces of rose and teak 
woods, and consider their weights and the colour of wood as 
features, then we might notice that rosewood is often 
darker than teakwood and also heavier. Thus we may have a 
rule for deciding the sample as rosewood If its weight is 
more than a certain critical value or its colour is darker 
than a critical value. If weight of wood can be measured 
by an index s^ , and the darkness by some other Index g^, 

"then we ca.ll fiie vpoinT’ <? _ T i T . 

X vector S ^ , Bg | as input f eature 
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•vector. Taking several wood samples and measuring the 
feature vectors and plotting them on a graph, the repre- 

sentation might appear as in i'ig, 2.3, 

Fotice that the points corresponding to rosewood 
cluster distinctly as against the cluster of points corres- 
ponding to teakwood. Thus we can form a rule of decision 
that if a feature vector occupies a position above a line 
separating the two clusters then we can decide the sample as 
rosewood, and on the contrary as teakwood. In other words 
the problem is to partition the feature space into two regions, 
where all the points in one region correspond to rosewood and 
the other region corresponds to teakwood. In the above case 
we considered only two features. We may be faced with the 
situation that a sairgjle of rosewood exhibits the same features 
as that of a sample of teakwood. In this case we might like 
to introduce another ground of classification, for example, 
the straightness of grain. Thus in order to increase the 
accuracy of classification we would like to include several 
features. In such a case the partitioning could be achieved 
by means of a hyperplane. When the set of points can be 
partitioned by means of a tgrperplane, then we call these 
noints as linearly se-parable . and the classifying machine is 
called a linear machine . If, in order to partition the 
pattern space more than one lyperplane becomes necessary. 
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then the samples are piecewise linearly (or nonlinearly) 
separable . The classifier is called a piecewise linear 
machine 4 The rule of decision and the mathematical expression 
describing it in feature space is called a Discriminant 
function . 

2.5 Some Basic Ide.ae in Design of learning Control Systems : 

In the previous section we examined the foundations 
of self -organizing and learning systems. In the present 
section we shall examine how these ideas can be used in 
designing learning control systems, 

A self -organizing system contains the following 
four basic subsystems, as depicted in Pig. 2.4s (i) Sensors, 
(ii) Preprocessor (iii) Learning network (iv) Goal circuit. 

The sensors observe the environment and provide 
the descriptive data for the learning network and goal 
circuit. The preprocessor processes the data from the sensors 
so as to match the requirements of leamiig system, e.g., the 
input state variables may be quantized and then these qiB.ntum 
states may be coded in a convenient form. In this way the 
complexity of the problem is reduced as only a finite number 
of quantum states have to be considered. The learning 
network is a variable part of the self -organiz ing system. 

It may be a device which changes its function chemically or 
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physically, or it may just he a part of a computer algorithm. 
At present only the first and the last type of learning 
networks are comprised of decision elements which operate on 
processed data and render a desirable response. The goal 
circuit directs the system organization toward a specific 
objective and provides infoimation on the degree of success 
attained at the end of each trial. This is usually achieved 
by one of the following techniques. 

(i) Reinforcement: If the present performance 
is an improvement over recent past performances, then it 
generates a reward signal to reinforce those states of the 
learning network that constitute the improvement and on the 
other hand weakens those states due to which the present 
performance gets deteriorated by generating a punishment 
signal. 

(ii) Error-correction; A correcting signal is 
generated by the goal circuit to weaken those states of the 
learning network only when the present performance is worse 
than recent past performances. 

At the heart of learning systems there are four 
basic concepts, referred to as artificial intelligence 
techniques. These are, (a) Mapping (b) Control -situations 
(c) Memory (d) Subgoal, We shall briefly describe these. 
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Mapping ; The mapping concept permits one to view control 
system design problems as collections of mappings from the 
state space to the control category space. Such a concept 
allows us to formulate the design problem in a form which is 
suitable for pattern recognition interpretation. 

Control situations : Control situations are regions in the 
state space for which a single control choice (e.g. a control 
+1 or -1) leads to satisfactory performance for all points 
contained therein. Such regions are obtained by pregridding 
the state space (e.g,, quantizing the state variables). The 
basic idea is that neighbouring points within a small region 
should have the same control choice with high probability. 

Memory : With the memory concept a separate memory location 
is associated with each control situation. As learning 
proceeds, it is necessary to store the pertinent information 
about the control situations encountered in the past. Thus if 
a particular control situation encoiuitered in past occurs 
again the learning continues' from the learned state attained 
in the past. In other words with the aid of memory during 
the learning process the learning system takes advantage of 
the past "experiences", and thus tends to improve its 
future performance. 



Subgoal : Often the main goal or objective for a control 
system can not be formulated precisely., or if it can be, the 
formulation is mathematically intractable, A common approach 
is to formulate a subgoal that approximates the actual goal. 

2,6 The ADALIME and (Ti-ainable Controllers : 

The Trainable Controller learning control system 
is one in which the inputs ani plant are incompletely specified 
The actual controller (Teacher) may be a set of control 
situations with a known crontrol choice for each. The learning 
system "observes” the activities of the "Teacher” and tries 
to copy them, i,e, for each control situation it tries to 
generate a control using its past "teachings" and compares 
them with that of the teacher. If the performance is bad 
then it changes its organization in order to improve its 
performance. In Fig. 2,5 a Teacher-learner Trainable control 
system is shown. 

The "Teacher” controls the plant by generating 
optimal controls for a small set of initial conditions X(0) 
of the plant. While the "Teacher" controls the plant, 
learning system learns the optimal control law. learning 
occurs with the switches SI and S2 in the position 1 and 1’ 
respectively. After learning is complete switches SI and 
32 assume the position 2 and 2’ respectively. In this mode 
the learning system controls the plant. 
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The basic building block of a learning system is 
an ADAlIl® ( ADAn tive Linear MEuron) . This is a simplified 
hardware copy of a biological neuron. Its structure was 
proposed by Widrow in 1962, A representative diagram 

of an Adialdia® is shown in Pig. 2.6 . 

The consist of a relay unit R, a summing 

device S, variable weights w^, w^ ... w^a,«kcl an adaptation 
machinery. The numbers ... are real numbers and a set 
of such numbers is called a pattern or stimulus . It is 
as sinned that the elements of a stimulus appear at the 
specified terminals at the same instant. The pattern elements 
may take values (0,1) or (--1,1), The adaptation machinery 
can vary any weight independent of the other weights. 

We describe a threshold device by, 
n 

js. .w. >■ T , then output response R = +1 '' 

i=1 ^ ^ 

- ( 2 . 1 ) 

n 

jg..w. T , then output response R = -1 

i=1 ^ ^ / 

! 

' ' ' ' * 

where T is a number called the threshold, are elements of 
the stimulus and w^ are the weights. 

Thus in order to get an output +1 the weighted 

:• ■ ■ : ' I 

sum of the inputs should exceed a threshold value. Hote in | 
Pig. 2.8 that there is a weight w^. By adjusting this 
weight the proper threshold value is obtained. Hote too that 






the input to this weight is +1 . If we replace T by w^ in 
(2.1) we obtain 


n 

i=1 

V 

O 

• 

then output 

n 

i=1 


then output 


( 2 . 2 ) 


In vector notation we have 


W 5 ^ * then output = 

T 

W ^ ^ f then output = 

The vector equation, 

W^3 - w = 0 

~ — o 

has a special significance, since it represents a hyperplane 
in the pattern space. A pattern vector S satisfying (2.4) 
lies on the hyperplane. Por the patterns not lying on this 
hyperplane, value of the left hand side of (2.4) is either 
positive or negative, depending on which side of the hyper- 
plane the pattern lies. In other words the hyperplane 
represented by (2.3) partitions the pattern space into two 
regions. 

Modifing (2.4) as 


+1 

-1 


(2.3) 


(2.4) 
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or in vector form, 

. m 

= 0 ( 2 . 6 ) 

where W and S are such that 

W = fw w. ... wl^ 

— L o 1 ry 

and S ~ '^2 * * * ^ 

For fixed weight settings there are 2^ different 
combinations of the input pattern elements with binary values. 
Each of these patterns would cause a +1 or -1 Adalihe input. 
Thus all possible input patterns are classified into two 
categories . The input-output relationship is determined by 
the choice of weights. We can imagine each pattern input 
being represented as a vertex of a unit-hypercube of (n+1 ) 
dimension. Thus adjustments of the weights of Adalihe in 
fact correspond to orienting an n dimensional hyperplane 
passing through the origin. Only a subset of 2^ input -out put 
relationships are however realizable by an Adaline . The 
set of patterns which can be classified correctly by an 
Adaline are called linearly separable patterns. 

Considering (2,6) we have the following rule of 
decision. If , 

W S ^ 0 , then the class of pattern = +1 - (a) 

(2.7) 

<. 0 , then the class of pattern = -1 (b) 
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Forming a matrix A by combining the rows of all 
input patterns which are linearly separable, then for a 
ij articular partition we have 



vvhere P"*" and P" a.re associated with the two classes of 
vertices defined by (2,7~a) and (2.7-b) respectively. The 
■lumber of patterns belonging to + class and - class are 1 and I 
m respectively. Hence, 

■■ ■ ■ i 

1 + m < 2^^ (2.9) 

We notice that ^ ^ ^ ^ ! 


P'^W 0 

and P”W << 0 

where 0 = f 0 0 



(2.1j 





If we multiply all the entries corresponding to 
P“ in the matrix A by -tj...-then we have 




A = 



and 


+1 


^ 1,1 


. s 


l,n 



-1 -n 


l+m,1 ' ■ ’ “■^l+m,n 


( 2 . 11 ) 


A.W > 0 (2.12)' 

let us assume that we know a closed form solution ! 

, ( 

for optimal control in a time optimal control system, i.e. 1 

I 

we know a switching surface in the state space described by ? 
the equation ! 

'i 

5' (2C-] j ^2’ • ' * = 0 (2.13): 

where , x^ . . . Xj^ are the state variables . ; 

Now if (2, 13), does not contain any cross product | 
terms then we have, 



0 


(2.14) 


P(X^, Xg ... Xj^) = S f,(x.) = 

If we quantize each state in q finite quantum 
levels, then we quantize the state space under consideration 
in q^^ quantum control situations. If we assign a code to 
each quantum level of each state then (2.14) becomes 

k 

X2, ... x^) = = 0 (2.15) 

where X^ is a vector representing a coded quantum level. 

Strictly speaking (2.15) approximates the original 
switching surface due to quantization errors. We can ignore 
this error since it can be made arbitrarily small. 

Pig. 2,7 shows a modified form of an Adeline along 
with a quantizer— encoder for each state -variable. It should 
be noticed tha.t it does not make any difference whether the 
weighted sum of all the patterns is taken or the weighted sum 
of individual state patterns is taken first and then the 
final sum is taken. The new indi-vidual threshold weights 
are such that the sum of these equals the original threshold 

weight. If the linear element has to mimic the actual time 

. . 

optimal surface , then in each quantum control situation at 
least at one point should be considered of the form i 

■ ■■ : I 

= Si’^ssi (2.16) 

Now let Q be the code matrix the rows of which are the codes 
assigned to the quantum levels of any state. 





« 








Thus if the m imi cs for all quantum levels 

of the state variable x^, then 

^ w (2.1'J'^ 

where *= [’^1^— **•• 

Now in (2.17) if <3 has rows which are linearly ‘ 
independent, then exists, hence we have 

% = t ( 2 . 1 ^^ 

In other words if the codes are linearly indepen- 
dent, then a solution for exists provided no cross terms 
exist in (2. 13) . 

The other criteria applicable to the realization 
of switching surfaces have been developed by Berkoveck and 
Epley [t^S j« These criteria are in terns of a property 
called projectability, which is defined: A surface that is 
continuous and single valued along any line parallel to a 
direction in space is pro jectable along that direction. 

Two tests for this property are: 

(1) A surface in n-dimensional space with a continuous 
boundary, symmetrical about an (n-l) dimer^ional hyperplane 
is projectable along a direction perpendicular to the 
hyperplane if the boundary is single valued on one side 
along any line side along any line normal to the hyperplane 
of S3rmmetry . 
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(2) A surface in n-dimensional space with a monotonic 
boundaiy, i.e, for which 

r . : 

■ ,.2^ 0 or ^ 0 , for any i s= 1, ... k 

cXf dx^ 

is projectable* 

The theorems proposed by Beifcoveck and Epley are: 

(a) A surface which is projectable may be approxi- 
mated with arbitrary accuracy with a single AdJsilaln^ by using } 
a linearly independent code for each of the state variables. 

(b) Any region which may be transformed into a 
pro jec table region with a linear transformation is realizable 
with a single Ad^alinia . 

These are sufficient conditions but not necessary. ] 
Finally we may point out that the criteria stated 

i 

above needs the analytical expression or the geometry of the 
switching surface. If, however, these are known, off-line- 
training would be of little xise. ! 

1 

■ I 

2.7 Storage Capacity of Adalihe t j 

From the last section we observe that there are 
^ 2^ possible patterns for n-dimensional patterr^. There are 

pH 

■ 2 "^ possible partitions of the 2“ patterns in different ways. 
Seveial attempts tei.ve been made to find out a relation to 
calculate the number of linearly separable partitions out 
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of 2 maximum, Attenpts hare been successful upto n *= 8 ' 
(Winder [if]). An upper bound has been estimated on the 
number of linearly separable partitions as, 

A/C .«n n 

^ '2.19) 

where n = dimensionality of patterns., 

In case of an inconQjlete training san5)le being 
specified (i.e., 1 + m ^ 2^ in (2.9), the curve in Fig, 2.8 
indicates the probability of realizing a randomly selected 
partitioning hyperplane. The abscissa is normalized and 
represents the number of nattems in the ssuaple per adaptive 
weight i.e. (1 + m)/(n + 1) . The ordinate Ps is the 
priori probability of achieving a randomly selected parti- 
tioning hyperplane of n-dimension, when using an Adkllffli with 
(n+1 ) variable weights. 

We notice that for large n a threshold for Ps 
occurs at (1 + m)/(n +1) = 2.0. This point is defined as 

the noimalized capacity of an it means that for 

large values of n, an AdLaline is almost certain to learn 
the classification of patterns in number lees than 2(n4-l) 
and almost certain not to learn patterns moire than 
patterns being selected randomly. 
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Eig. ■2. 8 formalized storage capacity of an Adeline. 






2.8 The Trainable Controller: 

As noted earlier the response of an AdaHne is very 
similar to a time-optimal controller (i.e. bang-bang type). 
Accordingly it is readily used as a trainable controller for 
time-optimal control problems, A representative block ■ 
diagram is shown in Fig. 2.5. 

The trainable controller consists of an Adallhh 
and a quantizer-encoder unit for each state variable as 
shown in Fig. 2.7. The teaching controller controls the 
dynamic system and supplies the ’desired’ control input u* to 
the dynamic system and the Adalinh during the tiaining mode, 
with switches and as shown in Fig. 2.5. The iid-dl3)34e 
learns the time-optimal control law and once the learning 
is complete the trained controller can be put to use with 
switches and S 2 in position 2 and 2’ respectively. 

2 . 9,1 Quantization-encoding of state variables; 

For the purpose of illustration let us consider 
an arbitrary switching surface with two state variables, as 
shown in Fig. 2,9. The switching curre - shown in Fig. 2.9 
could arise in a typical time optimal control problem.^ 

After the state variables X and T are quantized 
into a finite number of quantum levels as shown in Fig, 2,9, 
the state space under consideration is divided into several 





To use an as a trainable controller for 


fuel and energy optimal control problems, a network of 
jfiaaliiia? would be needed in general, since fuel and energy 
optimal controllers are multilevel in nature. 

The quantization-encoding of state variables for 
fuel and energy optimal control problems is similar as 
disdussed above. 

For a fuel-optimal controller we need a mi n-i mim 
of two Adallnh:- whose responses form a two bit word which 
can be decoded to identify the correct level i.e, +1, 0, -1, 
In case of energy optimal controller, the optimal-control 
can be quantized into n qiantum levels as shown in Fig. 2,10. 
Thus in case of energy optimal controller a minimum of 
(n-1 ) is needed for n quantum levels of control 

signal. We shall describe in detail the use of AaaliJfflS 
for fuel and energy optimal control problems in Chapter 3, 
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CHAPTER 3 

APPUCATIOHS Qg lEABHIHG SYSTEMS 
TO OPTIMAL G PETROL PROBIEMS 

3,1 Introduotion to Earlier Work ; 

In the pluvious chapter we discussed the method of 
learning control systems in obtaining solutions to the 
problems posed in Chapter 1. We noticed that when an optima 
control problem is viewed as a pattern recognition problem, 
the concepts of artificial intelligence can be usefully 
employed in synthesizing optimal controllers. We will notiC' 
that by the use of suoh techniques a controller so designed 
would be a quaai-optimal controller, since the switching 
surface it would realize, would be an approximation to the 
actual optimal switching surface due to quantization errors. 
The performance of such a controller would be somewhat 
suboptimal but, we find that its implementation would be 
more feasibleii 

Earliest work in this area is due to Smith [^20 J w.io 

studied the use of trainable controllers with threshold 
logic netwoiks for third and fourth order, time-optimal 
systems. In this study the state space was quantized into 
hypercubes and those with same input control description 
were collected by using a network of threshold logic 



61 


elemep-ts. The designed controller was not tested for 
generalization as all the hypercubes were treated as training 
samples . 

Subsequently Smith [ 2 %~\ has also considered the 
conditions for the realizability of switching surfaces. The 
analysis was carried out for the quantization-encoding of 
featiire variables. It was shown that the sufficient condition 
for an Adaline to realize a given linearly separable dicho- 
tomy is that the quantum hypercubes be encoded with "linearly 
independent code" . 

Mendel [2^J ha® made a detailed study of feasibility 
of using trainable controllers in optimal control systems. 

In his study, the system under consideiation was a simplified 
model of a time optimal third order re-entry vehicle. The 
system dynamics being as follows. 


X(t) » A X(t) + b u(t) 


where 


and 



t = [o 1 . 1? 





62 


u(t) 9. scalar control input with the 

constraint 

iu(t)| < 1 

The pjrohlem was to design a controller which would 
drive the ^steti from an initial state X(0) to a final state 
X(t^) * 0 in miiiiJDum tiine. 

The space of interest was quantized into 8000 

hyperoubes witti each state variable being quantized in 20 
quantum levels, code used was a binary, linearly 

independent cod® 'with elements +1 and -1 . A set of training 
samples was generating optimal trajectories in 

V 

the quantized ^tiate-spaoe with several initial conditions, 
uniformly distiri^'^'^®*^ state space. The size of the 
training set w^® 600, the percentage being 7.5% of the total 

hypercubes, 

A leaoTf^^^ controller was simulated on digital 
computer, whic]::^- trained on the above mentioned training 

set. It was fc?'*^<i that all the training samples were not 
classifiable by' controller. Mendel points out that this 

could be due tc? the switching surface being nonprojec table. 
It was found that the percentage of correctly classified 

samples fluctu^'^®^ ®^®^*^ 

The terminated after no more 

improvement wa^ obtainable and. after that the trained 
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controller was put to test on initial conditions not 
included in the training set. The controller was found to 
work satisfactorily. 

The application of trainable controller up to date 
has been limited to two level controllers (i.e. bang~‘bang) 
only. The reason for this is due to the difficulties assocx 
ated with obtaining training samples for multilevel contro- 
llers. Often two point boundary value problems must be 
solved; such problems become increasingly more difficnlt and 
computationally time consuming as the number of control 
levels increases. 

Elaborate system simulation may be required in 
order to generate the necessary data for the training set. 

It has been suggested in an application study in which the 
feasibility of realizing a switching surface model for ^ 
man-in-the loop was investigated (Mendel |^ 25 ] ) ; to let ^nan 
be in the loop to assist in obtaining training samples. 

The problem of obtaining data for training is 
common to all pattern recognition problems. Often such data 
must be obtained as a part of a different and even larg®^ 
study, Eor example in order to train a device to recogh-iZ® 
lunar features, HASA designed and launched a special 

Surveyor spacecraft to obtain representative photographs 

of these features [29]. 



WT 


In W Mendel's work the pe^ 

mance of the learning controller with one particula- 

. T our knowledge 

was studied (i.e. with elements +1 and -1 ) . o 

no study has been yet made in order to investigate 

convergence rate of training and the generalization^ 

j We w ould 

teristics of various linearly independent coaes. 

ice of code 

like to point out in this connection that the c o 

thP training » 

largely determines the convergence rate oi 

well as the generalization characteristics also ay 

, ig the 

the code used. Another point regarding the coae 
effect of coding quantum levels in different ways 


given code matrix. 

. large 

It is not unusual to be unable to obtain 

f learning 

training set. It is desired that the response o 

___ irainirg set 

controller should be favourable even if a smaii 

+ ”1 ^ ^ 

is available and the training set belongs to a 

region of the state space. 

The objectives of the present study are 

following; 

j rt-pfects the 

(i) to study how the choice of code ai 
.convergence rate of training and the generalizati 

characteristics, 

(ii) to study the behaviour of learnxnS 
when quantum levels are coded in different wa.ys 
given code matrix. Q, 
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(iii) the generalization characteristics of a 
learning controller for different codes when the training 
set belongs to a small region of the state space of interest. 

(iv) to study a different learning algorithm 
called, "Relaxation Algorithm", and its convergence charac- 
teristics. 

3.2 Introduction to Different Codes : 

It has been pointed out by many neurologists that 
high reliability in brain functioning is achieved due to 
redundancy in neural networks. Every neuron has a large 
number of inputs or synapses (as large as 100,000) and a 
particular neuron does not "fire" unless there are a largo 
number of "live" inputs. It has been noted that even if 
some neurons are destroyed or damaged, still the brain works 
fairly well. It would be interesting to use this idea of 
redundancy in case of learning machines, i.e., if a particular 
stimulus activates a large number of weights instead of a 
few, then during training the relevant information about the 
training sample would be stored in a distributed form in 
weights, and thus failure of a few weights would not destroy 
the "organization" of the machine. It is suggested here 
that this could be achieved by using codes having redundant 
structure.' 



In the present work, we have studied three types of 
codes. The code matrices have appearance as shown in Fig. 
3.1, 3.2 and 3.3. The code matrices have rows which would 
yield linearly independent codes, since the rows become 
linearly independent by increasing dimension by addition of 
a +1 element to each row (corresponding to the threshold 
weight). It will be noticed that the rows from the code 
matrices of Fig, 3.2 and Fig, 3.3 have two distinct parts of 
different character (i.e. 0, 1 and -1, +1 respectively). 

Fote too that the rows from the code matrix of Fig, 3.1 have 
only one element which has a different character (i.e. 1 as 
compared to zeros). We shall call for ease of notation the 
codes obtained from matrices of Fig. 3.2 and Fig. 3.3 as 
”01 contrast” and ”+1 contrast” type codes respectively. 

The code obtained from the matrix of Fig. 3.1 exhibits very 
little property of "contrast". Since its form is very 
similar to an identity matrix, we shall call the code so 
obtained as "identity" type code., 

.The codes described above are said to be in normal 
form. It will be noticed that there is a regular change o'bf 
.orders in rows as we move from the top row to the bottom 
row of any code matrix. In other words any row is very 
similar to the neighbouidng rows. However if the rows of 
a normal code matrix are shuffled in a random manner so 
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as to change the order of rows, the linear independence of 
codes is unaffected, and thus a "random code matrix" so 
obtained could be vexy well used for coding the quantum 

levels. A typical random 4*1 contrast type code is shown in 
Pig. 3.4. 


la^ivalence of differe nt linearly indenendent codas;; 


In the present context we will show that if a 
training set is realizable by one linearly independent code, 
then the same training set is realizable by using a different 



linearly independent code. also. 

Por a given code matrix q with linearly independent 
rows we have from "(2,17) 


where IQ is the augmented code matrix obtained by 
lion of a +1 to each row ofi original code matrix, 
t us assume that the 
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code matrix has linearly independent 
rows then a set of corresponding weight vectors wj_ exists 
and the same dichoton^ can be realized. 


new 


Effect of threshold 


xne aaairion oi a+1 element to each row of a code 
matrix has a special significance. This not only ensures 

the linear independence of codes but also serves the follow 
purposes. 


(i) In any pl^sical realization of an Adaii 
quantity (a.^) will be^nc^sy, 

the 'sign of #S ^"2 nSt give repeStabl 
performance for values of W^S near zero. Fn-r 
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It will be notioed that although the points of the 
two classes are separable by means of a straight line, the ‘ 
separating line does not pass through origin, and thus an 
equation of type y « mx will not satisfy the requirements of 
partition. It will be noticed that if we add an extra 















^ However If it is ieeired that the iiiput stimulus 

produce a markedly different response from S. , then the 
overlap of stimuli mi,ht create difficult,. Mdustment of 
the weights attached to S, so as to produce the proper 
response for will upset most of the weights which oontri 

. Ihis is called learning inter: 
Terence, and is similar to retroactive inhihition erperienc 

y hrclcgical organisms when presented with highly similar 

Stimuli for which different 

xon aiiierent responses are required 

I«amlr« interference can be. overcome by ^peated 

appUoatron Of a- IgasL iV! algorithm . Sepeated adaptations 

eventually lead to sufficiently large magnitudes being 

attatched tc the few weights which do not correspond to the 

overlap of the stimuli. In aumary the Adallne's ability to 

generalise is due to the overlap of stliuli. Ihus' fk.m the 

above discussion we erpeot, the Adeline fo generalise better 

■vath contrast type codes as collared to identity type codes. 

Smith [24] has shown ■that ge»SrAlizatloa'±sfeft^^^ 

ensfured after traininfr 



Smith also indicates that the Adeline has a strong 

tendency to generalize correctly even -i-p v 

even rf the above conditions 

are not met* * 

m formalisation of generalization criteria exists 
heyond that stated above. The usual approach to verify the 
presence of generalization in a practical problem is to test 
it by using samples not included In the original training set. 

learning Algorithms 

Ollhe learning algorithm associated wiih a single 

.daline, adjusts the weight vector W to realize a solution 

'f (2.3). Several methods of adjusting the weights exist as 
e tailed, 

A and B denote two classes. An output of +1 
rom Adeline means that the input stimulus Is recognized as 
member of A and on the oontraiy an output of -1 means that 
he input stimulus is recognized as a member of B. 

One method of adjusting the weightsis the forced- 

”«^^hod each input stimulus element 
> added to the corresponding weight if the stimulus is a 

.mber of A, and is subtracted if the input is a member of 
his method of learning has the disadvantage of being 
■pendent on the sequence of traiaing samples and of not 


4 »/'' 
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Second method of weight adjustment is the eifoi- 
correct ion-met hod . In this method the weights are not changed 
if the Adaline classifies a stimulus correctly. If the input 
pattern is actually a member of A but Adaline classifies it 
as a member of B, then each input stimulus element multiplied 
by a constapt ^ is added to the corresponding weights. If 
the input stimulus is actually in B but is classified as a 
member of A, then each stimulus element multiplied by a 
constant is subtracted from the corresponding weights. For 
correction of any error, remains at a certain positive 
value. Generally the value of is taken to be 1 . In the 
present work we have considered this algorithm for =1. 
Error correction guarantees a solution for (2.3) if i^is 
chosen to satisfy certain conditions apd the training set is 
linearly separable. The proof of this method can be found 
in lilsson[26l, Block [_34l and Rosenblatt [31]* 

Another method of adjusting the weights is the 
Widrow-Hoff-'iidnimiiatti^mean^sauare algorithm. In this algorithm, 
the error for an iiiput is defined as the desired output minus 
the weighted s-um of stimulus elements. At each presentation 
of an input stimulus, each stimulus element times the error 
times a constant is added to the corresponding weights. 

It has been shown that if the mean square error is a 
hyperparabolic function of the weights, then this algorithm 



IS equivalent to searching the minimum of the hyperparabola 
using the method of steepest desoent. This algorithm tends 
to cluster all input sums for members of A, around +1 and 
all input sums for members of B around -1 . it is possible 
to modify this algorithm so that the weights are adjusted 
only if the Adeline output is wrong {i.e. error correction ). 

Another method of changing weights is the relaxation 
algojjithm . This algorithm is stated mathematically as 
follows. 

let W(k) be the present weight vector. If an error 
occurs, the weight rector after oorrection will be W(k+ 1 ). 

The rel£(:^atioii fule is 

W(k+1) ^ W(k) S(k) 

" f|S(k)p 2 ’ 

if w’^(k) ^(k) < 0 ■ (3_5) 

and 

W(k+1) = W(k) , if W^(k) S(k) > 0 

where ^(k) is a row of A in (2.12). 

This algorithin has a simple geometric meaning. 

The quantity, 



is the distance from the tip of vector W(k) to the hyperplane 
W'^(k) S(k) = 0. Since S(k)/ j| S(k) !l is the unit nomal vector 
for that hyperplane, (3.5) calls for W(k) to he moved a 
certain fraction X of the distance from W(k) to the hyper- 
plane. If A = 1, W(k) is moved exactly on the hyperplane. 

The convergence of the above algorithm, is guaranteed for 


0< A < 2 

for a linearly separable training set. The algorithm may be 
convergent for A >2, but convergence is not guaranteed. 

The proof is as follows. 

I«t W be a solution weight vector. Thus if the 
algorithm is convergent then with each correction the weight 
vector W(k) naist get closer to W* . Since 


W(k+1) 


, , I^(k) S(k) 

W(k) - X j ^ S(k) 

l|s(k)jl2 


we have 


II E(k+1) - W* II ^ W^(k) W(k) + 


■T, 


V(k) S(k) 


-l|S(k)|| 


,T 


S"(k) S(k) 
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W(k+1 ) - W 


W(k) - w 


2 , „ , fw S(k) 

'+ 2 A g ■ 


lls(k)!) 


[w" - w(k)j^ S(k) + 


w'^(k) SCkT^ 


(3.8) 


L || s(k)!| 2 

Bow since W/*8(k) )> 0 for any pattern S(k) from the 
training set, we have, 

[w* -W(k)J^S(k) > - W(k)^ S(k) >0 (3.9) 

from (3.9) and (3.8) we have 






-A (2 -■*>) 


[W(k)'^ S(k)]2 
. I|s(k)!l2" 

( 3 . 10 ) 


Thus from (3.10) we find that W(k)' gets closer to 
W (k) with each correction as long as X is restricted to 


the range 0< >^<2. In other words the sequence of weight 
vectors W(1), W(2)... gets closer and closer to W* and in 
the limit as k goes to a large value the distance j| w(k) - 
approaches some limiting distance r(W* ) . This means that 
as k goes to infinity W(k) is confined to the surface of a 
hypersphere with center at W* and radius, ri'g ) . Since this 
is valid for any solution weight vector W , the limiting 
W(k) is confined to the intersection of the hyperspheres 
centered about all the possible solution vectors. 
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It can be shown that the common intersection of 
these bgrperspheres is a single point on the boundary to the 
solution region |^2§j, (^26^, [2'7'[, 

This algorithm has been used in automatic classifi- 
cation of chemical compounds by Isenhour and Jurs [*28^, for 

A = 2 . 


3 .4 Learning with Two levels of Control: Time Optimal 
Learning Control System ; 

As noted earlier, an Adaline can dichotomize the 
feature space after being trained on a training set, the 
Adaline can readily be used as a time optimal controller as 
its response is bang-bang type. 

We shall first study the performance of learning 
controller for two dimensional case i.e. with two state 
variables, as it is easy to study the generalization 
characteristics in two dimensions. 

Before studying the actual time optimal control 
problem we shall briefly examine how an Adhline generalizes 
after being trained on a training set of two samples for 
different codes. 

A learning controller was simulated for learning 
two training samples using +'1) contrast, 01 contrast and 
identity codes for the two dimensional state space. iEach 

state was,;qi^ntized, i]|}|tp code was .. 
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assigned to each quantum level, coding being done with the 
normal code matrices oi Pig. 3.1 , 3.2 and 3.3. The learning 
algorithm used was floced increment learning method and initial 
weight lector was arbitrarily chosen to be zero. 

5'ig* 3.7, 3.8 and 3.9 show the generalization maps 
of the controller after being trained on the training samples 
marked. It will be noticed from these figures that the 
Adeline coded with the contrast type codes, responds to 
"unseen” patterns on the basis of having "seen” similar 
patterns during training. As noted earlier this is due to 
the effect of overlap or iptersection of patterns. On the 
other hand the Adeline coded with identity type code seems 
practically inert to new samples. 

Prom the above discussion it is clear that the 
controller using contrast type codes has a tendency to 
generalize over a large space. This property is particularly 

useful when training set is small and the training samples 
are widely separated. 


3:et us now consider the learning and generalization 

characteristics for a larger set of training samples with 
fixed increment learning algorithm. An arbitraiy time 


optimal switching surface was chosen as shown in Pig. 3 . 10 . 
The states (X,Y) were quantised in 40 .quantum levels each. 

A set of ^ fifty PPJSp3^,s..'was chestfeit; iPnir +.-v>Q-iinr? 
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Fig* 3.10 An arbitrary time optimal switching dtehoton^.ahd 
various zones formed due to a limited training 
set. 
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Table I 

set for a ti me optimal leamiaip Qfpnt-roller 


S. 

tsq . 


^ ■!— '»i— — ■ u fi m f , <M H yi _ ^ _ _ 

jx axis ;T axis { [ {X axis {Y axis • 

|quantum|quantma|Control» * {quantun} quantum 'Control 
{number |number f | {number {number | 

■ —f" — i- ■■ -« « . 


1 

26 

12 

+ 1 

26 

30 

19 

-1 
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10 

32 

-1 

27 

12 

29 

+1 
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31 

”1 

28 

14 
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28 
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related to the noh-optimum movements of the 
hyperplane , , -AdalSj^ he^d^ -2% correotions 


The training samples depicted in Table I, were 
chosen from the zone marked A in Pig. 3 . 10 . The samples were 
chosen sufficiently close to the optimal switching curve. 
Control situations, through which the switching curve passed, 
were not taken as training samples# 

An Adaline was simulated and was trained on the 
above training set for different codes with initial weight 
vector as zBxot Training was achieved by piemen ting the 
training samples in the sequence as shown in Tdble 1. The 
learning curves of the Adaline fof different codes are shown 
in Pig. 3.11* It will be noticed that the a priori probability 
Pg of correct diahotomization rapidly inCreasds and finally 
attains a value uhity, indicating that the switching surface 
is projectable. It will be also noticed that the convergence 
for the Adaline using +1 contrast type code is rapid' as 
compared to other codes. The learning curve follows (roughly) 
an exponential curve and Pg fluctuates about a value close 
to unity for large values of number of corrections and finally 
converges to unity. These fluctuations are observed in all 
experiments performed with this algorithm and' seems to be 
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of the Adaline are shown in Pig. 3.12, 3,13 and 3. 14 for 
various codes. It will be noticed that the controller has a 
very good generalization characteristics with contrast type 
codes, whereas it suffers from poor generalization with 
identity code. ITumber of misclassifications in case of contrast 
codes was very small in the zone of knowledge. The behaviour 
of Adaline in the zone of ignorance is not favourable. This 
is due to the lack of training samples from this zone, and 
improvement in performance could be achieved if some more 
training samples are included in the training set from this zone. 

In order to investigate the generalization charact- 


eristics with different codes, the Adalines using random codes 
of Pig. 3.15(a) and 3 . 16 (a) were trained on the training set 
of Table I, and tested for generalization. Pig. 3 . 15 (b) and 
3 . 16 (b) 'depict the corresponding generalization maps. It will 
be noticed that the generalization is badly deteriorated. It 
was observed that the training took a larger number of itera- 
tions. The Adaline needed 543 corrections for +1 contrast 


code and 2306 corrections for 01 contrast code. 

In order to study the generalization characteristics 
in greater another tiainii^ ah' "shown ‘in 

Table II, The training sample^ were now chosen from zones 
and G^. ThO general^lzaticm eharaejfj^|ics of the ' controller 
for the three normal - Pf® ■ ... ^ 1 v--. -'z i a 1 n 
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Table II 

An alternative training set for training 
time optimal learning controller 


, ■ - - ^ ^ T- - — ^ ^ 

„ {X axis ,'Y axis J { s axis {Y axis } 
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The niimber of eor-rpn+-; , 

oorrecticms needed for various codes 

were 377. 1300 and 177 for ddaXine usin, nor« ideniif,, 01 

oontrast and . 1 oonfrast codes respectively, if ^ 

noticed that oontrast ^pe codes have a strong tendency to 

generalize correct 3 y in the zone of v . 

partial knowledge and 

zone of ignorance. This pronertv -ic 

^ property is reiy prominent in case 

of +1 contrast codes (Pig. 3.I9). 

. that the flnctuation of a 

prrorr probabllrty of correct classification after a large 

number of corrections could be due te non-optimum movements 
of the partitioning hyperplane m fixed increment error 
correction method. The relaxation algorithm is a special ‘ 
form of error correction method in which the correction 
coefficient is variable between thp c.ncr.d. 


and 


WtttD = W(h) -r^S(h), if wV)SW< 0 


w«k+i) = w(k) 


if W (k) S(k) ^ 0 ( 3.1 


where 
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learning process to converge faster. We shall briefly examine 
the convergence characteristics of this algorithm for different 
codes. Table III shows the training samples of Table I, but 
in a different sequence. It was found during the experiments 
that the sequence of training also affects the convergence 
rate. Keeping this in mind another training sequence was 
also tried in which the training samples were presented in a 
random manner, i.e* a random number between 1 to 50 was 
generated and accordingly the corresponding training sample 
was presented to the Adaline, Graphs were plotted to investi- 
gate the nature of convergence with respect to . As shown 
in Pig. 3.20, 3.21 and 3.22, the convergence is faster in 
case of random presentation of training samples as compared 
to sequential presentation. The convergence is extremely 
fast in case of +1 contrast code, and further for training 
set of Table I, Adaline using 01 contrast code exhibits 
discontinuities at A =1.0 and 1.5, nonconvergence being 
obtained even after 30 minutes of computer run. It will be 
noticed that in Table Illj the training samples of the two 
classes are mixed up as compared to training samples of 
Table I. This suggests that for faster convergence of 
training, presentation of training samples should be random. 
Generalization characteristics of controller trained with 
this algorithm were very similar to that of fixed increment , 
error correction learning algorithm. 
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Table III 



! I, , i r t t 

g |X axis |Y axis } ' S axis |Y axis J 

Jquan'tuiitjq.uanttlm’ Control} [quantiim} quantum {Control 

{number {number { { * {number {number { 
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3.5 Learning with TJaree Levels of . Coiatrol; guel OT^ti.mal 
Learning Control Systems ? 


As pointed out in Chapter 1 , the fuel opti&al 
controllers are of hang-off-bang nature; i.e. there are three 
stable states of the control. Since an Adaline can generate 
only two states of control* it is therefore necessary that 
the control be decoded by using a two bit word with : a 


ml m' Tmim of two Adelines .. f ' for a fuel optimal 

learning controller. Fig. 3.25 shows the sej;up to realize 
the fuel optimal learning controller. ■ 

Fig. 3 ^ 24 . shows an arbitrary fuel optimal switching 
dichotomy ^’elected for training the controller. The states 
were quantized in 40 quantum levels each and training samples 
were chosen fhom the quantized state space. !Iable IV shows 
the training seti^ 

Mendel [^2 ’ 9 ] has described a method of training for 
the fuel optimal learning controller shorn in Fig. 3.2'5. 

The method is as follows. 

Let Uj and Ujj be the responses of the Adaline I 
and II respectively, I^t US assume that we are having a , 



training set of patterns with known classification, lie 
E = +1 , 0, or -1. We wish that a t#o bit word fren the 


Thus if we 


fMSM 
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Adaline I 



^ig* 3.23 


A network of Adalin^to reali^P 4.'- 

control law- ^anze a fuel optimal 






Table IV 
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A training set for trainiiag the fuel optimal controller 


S. 'IX' ^is 

IX axis 

1 

t 

t 

f 

s . ‘;x 

axis IT 

axis I 


Mo. Iquani7umIg.uantui)i|CoELtyol I 

Mo. I quantum I quantum. {Control 

Inumber 

{number 

t 

1 

't 

Inumber Inumber I 


1 2 

32 


+1 

44 

22 

17 

0 

2 2 

33 


0 

45 

22 

18 

0 

3 4 

33 


0 

46 

23 

15 

0 

4 6 

31 


+1 

47 

23 

16 

0 

5 7 

32 


0 

48 

24 

13 

0 

6 8 

30 


+ 1 

49 

25 

15 

0 

7 10 

31 


0 

50 

26 

14 

0 

8 11 

, 29 


+1 

51 

26 

11 

0 

9 12 

30 


0 

52 

27 

10 

0 

10 14 

29 


0 

53 

28 

9 

0 

11 14 

27 


+1 

54 

30 

8 

0 

12 15 

26 


+1 

55 

31 

7 

0 

13 16 

27 


0 

56 

33 

6 

0 

14 16 

25 


41 

57 

35 

5 

0 

15 17 

24 


+1 

58 

37 

4 

0 

16 18 

25 


0 

59 

36 

3 

+1 

17 18 

22 


+1 

60 

34 

4 

+1 * 

18 ■ 19 

23 


0 

61 

32 

5 

+1 

19 19 

24 


-0 

62 

30 

6 

+1 

20 20 

25 


-1 

63 

28 

7 

+1 

21 20 

26 


-1 

64 

27 

8 

-fl 

22 18 

27 


-0 

65 

24 

11 

+1 

23 19 

28 


-1 

66 

23 

12 

+1 

24 17 

29 . 


-0 

67 

22 

13 

+1 

25 18 

30 


-1 

68 

39 

9 

0 

26 16 

30 


'0 

69 

36 

10 

0 

27 17 

31 


-1 

70 

35 

10 

0 

28 15 

31 


■0 

71 

32 

11 

0 

29 16 

33 


~1 

72 

39 

12 

0 ' 

30 13 

33 


0 

73 

28 

13 

0 

31 12 

34 - 


-0 

74 

26 

14 

-0 

32 13 

36 



75 

39 

11 

-1 

33 11 

35 


0 

76 

34 

12 

-1 

34 10 

36 


-0 

77 

51 . 

13 

-1 

35 15 

36 11 

37 . 10 

38 7 

. 39 '8 

54 


78 

'4"'' 

»-"l ' 
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Fig. 3.24 Arbitraiy fuel op timl ^witching dichotoiEs?- 

selected for training the Adalineoont rollers 
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2 

+1 or -1, then there are 2 =4 distinct words that can be 

presented as input to the decorder. If we can assign a 

each 

distinct word to control level then the response of the 
Adalines can be mapped onto one of the control levels. 
Mendel [29] has suggested that if the control levels are 
coded as shown below in fable V, the network of Pig. 3. 
can be trained on ariy linearly separable set. 


Table Y 


O V-<- JL V./ J. WAO. W J- W -JU O o U 

Response of 
Adaline I 

! 

t 

t 

1 

t 

Response of 
Adaline II 

I 

t 

r 

1 ^ 
t 

Control level 

+1 


1 -1 ■ 



-1 ■ 




0 

-1 


' ■' -1 


-1 



The network of Adalines shown in Pig. 3-23 was 
simulated using the above described method and trained on 


the training set of Table TV. ' It was found that the method 











Table VI 


114 


Coding scheme for control levels 


Response of 
Adaline I 

} Response of 

Adaline II 

f 

1 

I Control level 

t.. - _ 

+1 

+1 

4-1 

-1 

+1 

0 

-1 

-1 

-1 


Another coding scheme which is also foiind to be 
convergent is shown in Table VII. 


Table VII 

Alternative coding schen^ for control levels 


Response of 
Adaline I 

1 

{ Response of 

} Adaline II 

t 

1 

1 

t 

1 

1 

1 

Control’ level 

+1 

-1 


+1 

-1 

-1 


'■ 0 

-1 

+1 


-1 

It will 

be noticed fr<mi Rig. 

3.2^ that the control 

levels +1 and -1 

are separated by the 

control level 0. 

Thus if training samples are chosen close 

to the switching 



>1 


surface then, th< 

stimuli chosen near 

•i^e 

au^ace A will 




.^1 't •' V y \- 

belong to either 



It gm3d';wili, baye 
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closely similar patterns. Similarly the samples chosen near 
the surface B will either belong to 0 or +1 control levels 
and will have similarity in patterns. Prom the mapping as 
shown in Table V the separating hyperplanes realized by the 
two Adelines will have a form as shown in Pig, 3,2$, It is 
clear from Pig. 3.2$ that the stimuli of the 4-1 and -1 class 
near the partitioning surface due to Adeline I should be 
similar, which is contrary to Pig. 3.2.i^. This will lead to 
the nonconvergenoe of the training since there will not exist 
partitioning hyperplanes to realize the training saHq)les, 

The fuel optimal learning controller of Pig, 3.25 
was simulated using fixed increment e®roiix*g>iec^t 3 :i)nr*algorithm 
Training was accoD 5 )lished on the training set of Table IV 
for different normal codes. Pig. 3.25, 3.2^ and 3.2B show 
the generalization characteristics for various codes. It 
will be noted that the generalization due to identity code 
is poor as compared to the contrast codes. In Pig, 3.£f7and 
3.28, poor definition of dichotomies near origin occurs due 
to lack of training sanqjles near origin. 


To investigate the generalization property of codes, 
random codes were also tried. It was noticed that with 
contrast lype codes, the controller usually failed to 






ISCt mm 

, . wsmii mmtt 




i i I [) D I 1 I ! S’ J J 

30033000030 3 O-l-l-l-l-l-l-l-l-l-l-l-l-l-i-i-x-f-i -Lf >£.[ 1 I l I i 

11130100000000 O-l-l-l-l-l-l-l-t-l-l-l-l-i-i-i-i-i I 1 i 
lillLlllOOOOOO -I -I -I -I -I -I _i .1 _t I 

iiiiiiiiitiiiooo ; ; f r J . . ! 

I 111 II iixiiiiiioo o-i-i 0 o-i-i-i-i-i-i-i-i-i-i-i -I -i -I -* -I -I _i 

lllLllilllLlillll, O-l-l-l-l-l-l-l-l-i-i -1 -i-i -I -l_i _i _i _l_i I I 

L 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3-1-1 a 0-i4-i-i-i-i4-i4-t44-i4.i444 

Lilli iiiiiiiiiiiiia 1 0 0 6“i-i-i-i-i-i-i,-i-i -i-i -I -1-1 -i_i ,1 

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 i 1 1 1 1 o-i-i-i-i44444444444444 

LllLLliliillllllllOai o-l-l-l-l-l-l-l-l-i-i-i-i-i I I r i I 
till III liltll 1111 3-1-1 0 0-l-l-l-l-l-l-l-i-i-i-i-i_i ^|-i.iZi_i 

I I j 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 o-i-i-i-i4-i-[-i-j-i.i444444 

ll \ l } 1 1 1 1 i \ \ ^ ^ ^ ^ ^ ^ ^ ^ ^ o-i-i-i-i4-{-}-£-i.i-i444-i44 

f } f f } f J H n J J H ^ ^ ^ ^ ^ ^ ^ ® ® ° » 5 0 0 3 3 0 3 0 0 0 0 I 

iiiiiiiJff f \ } lllllllll 100 0 0000000300 f f I 

4 J f M } J } i n f ^ ^ ^ ^ ^ ^ ^ ^ 1 I 1 1 1 1 1 2 0 0 i a 3 ^ 0 I f I 

I i i I i I i 1 I f I i I } J J ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ t ^ 0 I i 

I II lllllllll 1 I, lli I 1^1 l.i-A.vi I I ’i''a I'ti i’ 

1 i 1 1 1 M } 1 1 M i I M } i V i I i i i i » i ill [ J f } M I } ^ 

t I i I t I n I t I I I I I H I X 


i&iMim 619% &i 







"l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
-l-l 
6-1 
O-l- 
3 - 1 - 
3 3 
3 3 
3 3 
3 3 


-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 


-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l- 


-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

-l-l-l-l 

l-l*l-l 


l-l-l-l-l-l-l-l 


-l-l 

-i-l 

-l-I 

-l-l 

-i-l 

-l-l 

-l-l 

-l-l 

-l-t 

-l-l 

-l-l' 

-l-l- 

-l-l- 

-l-l- 


-l-l 

-l-l 

-l-l 

-l-l 

-l-l 

-l-t 

-l-l 

-l-l 

-l-l 

- 4-1 

-l-l 

-l-l 

-l-l 

-l-l 

-l-l 


-l-l-l 

-l-l-l 

-l-l-t 

-l-l-l 

-l-l-L 

-l-l-i 

-l-l-l 

-l-l-l 

-l-l-l 

-l-l-l 

-l-l-l 

-l-I-l 

-l-l-l. 

-l-l-l 


O-l* 
O-l- 
0 0 - 
0 D- 


l-l- 
l-l- 
l-l- 
1 -! - 


l-l-l-l- 

l-l-l-l- 

l-l-l-l- 


l-l-l-l-l-l-l- 


- 1 -? 

-l-i 

-l-l 

-l-t- 

-l-l- 
-l-l- 
-l-l- 
-l-l- 
-l-l- 
— I — I — 
-l-l- 
-l-l- 
-l-l- 
-l-l- 


l-l 

t-l- 

l-l- 

t -t - 


l-l 

l-l- 

l-l- 

1 _ i _ 


l-l-l 

l-l-l 

l-l-l 

ft. ^ B S 


l-l- 

■l-l- 

■l-l 


-l-l-:. 
-l-l-' 
-l-l-l 
-l-l-l 
-l-l-l 
-l-l-£ 
-l-l-l 
-l-l-l 
- 1 - 1-4 
-l-i-l 
-l-l-l 
-l-l-l 
-l-l-l 
-l-l-i 
l-l-l 
l-l-l 
t-l-l 
l-l 





- — • ■ ' •• ' ’’ ' ^ ~ ”<*•, “i. ” “ 1 . ,i J 

.... ‘ i I 1 1 i; . 1 , . ^ 

- J i i. i . i A , I A i . 

. 'mm ■ mum mm ««. ‘ *.' * *'1- ■*«*** J,.» *“ ^» ** ^ <4. i.^- A i I f 

^ _ I I ^ 

-.-ii i-’rrz"z Z”~j~''*'v 1 i 

' _:.^Z"ZC-:'"‘^~*‘'T’‘'^“^^"^^'“^“"~^”^“ -i-i”i-..- 

-' - t : f- J M J ^ ^ i i i i i i 1 j. I I i 

' ' ■ ‘ * ;■' - " “'•-i~i-3--i.“x iiij. iii,xi'‘ 1 1 1 ' 1 

- , -,- -;.5-:-;_ _L"_tZ'Z^‘'^i ^ ^ ^ i * i ^ 1 1 a 

-t . ^ t 1 * 1 I I 1 ^ » 1 I 

■ ' -■ ■• , ‘ ‘ ■■ .'• r* ■)■“'! ; 1 * i i. > 1 i . i 

_ ■ .' -■' “' " " *“ *•■ i & * A i i A i i i i- 1 i 1 ' 1 ^ 

. J .^. ^ X i I I A X ^ ^ 

z,-xz _-,-.z:z^.Zf"f";-J1-^-f^ i I I I I if 

Z'’Z‘''Z i i 

*" ' «.| 1 ^ f 

„ . . „ '* • » ^ «a iii* «;* * i 

■“ A X "* I *** ;, •*' Z ***'7 *^7 ”"X ***X *"1. X 

* 4 ‘ r ^ i»7* A 

-A-l- ~ 


.J- J 

i k 

.’• i 
1 A 

•'L-L- 

-i-S- 

■i-i- 

I I 
I I 
I i 

i 

i I, 

.1 i 




^'a-g. 3.3© and 3.3CI demonstrate the contrast random code 
matrices for which the controller convexEes to a learned 
state. The corresponding generalization maps are shown in 
^’ig. 3.35 and 3.3‘3. 

From the previous discussion it is apparent that 
a "good" generalization is guaranteed if the normal contrast 
type of code is used to code the quantum levels. A faster 

convergence of the learning process is however achieved with 

±1 contrast type of code. Improvement in convergence is 
obtained by the use of relaxation algorithm. It was also 
notreed that there is no significant difference in generali- 
zation characteristica with the fixed increment and relaxation 
algorithms. Improvement in convergence is achieved with 
random presentation of training samples. 

With this information in hand we shall investigate 
the characteristics of a 3^ order time optimal controller. 
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and 


k - [ 0 


1 ]' 


Thi. is a simplified re-entiy vshiole model by 

Mendel 


Here the basic problem is to drive the system from 
an initial state X(o) to the origin in minimum time. 

The region of interest in state space was chosen as 
l?ll< 10 , I r2t< s, and <25. au the three states 
were quantized Into 40 quantum levels, thus forming 64000 
quantum control situations in state space. Enccdlng of 
quantum levels was done with normal codes. 


As previously stated, the open loop, time optimal 
control is the control u*(t) which satisfies (1.8) and (I.10) 
such that terminal time t^ is a minimum. Feustadt 3 has 
developed an iterative method to oon^jute the control u*Ct) 
to generate the optimal trajectories in state space. The 


details of this method can be found in Mendel [ 39 ]. 
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control situatlono were identified through which the optional 

trajectories passed. The computer program rejected redundant 

control situations (i.e. control situattcns already encoun- 
tered). 


The Adeline oontrollen using fixed increment emor 
correcticm method was simulated and was trained on the 
training set as obtained above. The training set included 
560 training samples forming 0.875?S of the total control 
situations. The training was found to be convergent for 
±1 contrast code within 1079 itexations although convergence 
for identity and 01 contrast codes was not obtained for 
iterations below 2000. This suggests that the surface is 
p rojectable . Training was terminated arbitrarily after 1000 
iterations for all oases of. codes. 


J’lg . 3.34 to 3 .42 are the typical time response 
charactenstios for initial conditions not included in the 
training set,, using different codes. It will be noticed 



^ xx-uia opTimaxity is small. The unreliable 
using identity code is very clearly indi- 
\-i 3*37 and 3*;40., It will be noticed too that 
the Adaline changes control very rapidly* 

^ (op-feuDal) and (i-daline control) shows 

come within about 
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Of the switehlr^ surfaee In the vicinity of origin and thie 
can be eliminatea by the use of finer ijuantication near the 
origin OP by swltohlng to a different node of control. 



— •arirer our idea of introdaoiog 

".dundanesp la oodee is siaila^to the fimotioa. of neurons in 
brain. Tests on trained Adeline were carried out for 
reaUability for different codes. After training was conplel 
the weights of the Adeline were reduced by using integer 
srithnetic for all three codes. After this the controller 
was put to generalization test as shown in Pig. 3.43 3^44 

and 3.45 for different codes. It will be noticed from these 
figures that for, the case of contrast codes the response of 
the controller is quite favourable, on the other hand the 
response of Adallne using identity code is badly deteriorated 
Table VIII shows the performance in response time of the 
optimal controller and the trained controller with noimal 

and disturbed weights. The response time for trained cent- 

roller is the time to dz*i vp . 

xme -CO arive the states ^within one quantum 

control situation of the orig'f n .v, 
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gable YITT 


Actual and optimal response 


Initial 

condition 

X(0) 


Code 


i {grained {grained controller 
ttime I controller {response time 


■{response time 
.response 'with 2095 disturbed 
.time {weights 


, ^ m ^Identity 

(— 6.0,0,0, 1 5»0/^ 01 conlirast 

+1 contrast 

20.34 

'19.94 

22.28 

23.09 

19.78 

21.53 

22.39 

( 5 • 0 1 3 • 0 , 0 • 0 ) 01 con'tras'fe 

+1 contrast 

10.22 

16.73 

7.29 

8.01 

21.53 

7.85 

7.85 

( *5 . 0 , 0. 0 , -1 2, 5)® 01 contrast 

+1 contrast 

18,04 

23.42 

17.12 

17.05 

23.38 

17.14 

16.90 

(4.1,3.2,11.0)1 Sf^ontoast 

+1 contrast 

IT, 63 

21.33 

16.81 

20.57 

20.66 

18.94 

20.37 

t A • ^ ~ .Ti Identity 

X 1 , 3 . 2, — 1 1 , o) 01 contrast 

+1 contrast 

15.36 

21 .71 

14.54 

14.76 

23.03 

14.50 

14.17 

(5.0, 0.0, 0,0) 01 contrast 

+1 contrast 

8.16 

14.27 

7-. 34 

77P03 

14.01 

7.85 

7.08 

ft- -rr- Identity 

4oxS^* contrast 

-14. 32) +1 contrast 

20.83 

28.37 

20.64 

19.96 

20.77 

19.68 



Table VIII continued 


( 4 . 1 , - 3 . 2 , 
- 11 . 0)1 


( 5 . 0 , - 3 * 0 , 
- 12 . 5 )'^ 


Identity 

01 contrast 14,62 
+1 contrast 

Identity 

01 contrast 18.67 
+1 contrast 


17.68 

24.49 

15.96 

14.11 

14,18 

13.70 

18.93 

25.66 

19.05 

19.03 

17.68 

17.64 



M H 


H2 

informa-tion is less exposed to -variation in weights in case 
of contrast codes, since Information during training is 
diffused in a large number of weights. 

The most important conclusion from the above work 
is that the learning controllers oan achieve a practical 
realization of the complete closed loop suboptimal control 
law with a small set of training samples. It is termed 
Practical for the following reasons. 

(1) Quantization need not be very fine 

( 2 ) The size of training set can be smll 

(3) Controller response is less affected by changes in 
weights 

(4) Exoessive amount of data processing tinie is not 
required 

Hardware implementation using chemical cells, 
integrated circuits etc. is straightforward 


( 5 ) 
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CHAPTER 4 

P0HC3US I0HS AHD SCOPE OP PURTHER WORK: 

In -the earlier chapters, we have shown that the 
application of artificial intelUgence techniques are an 
effective means of realizing arbitrary optimal switching 
surfaces. The advantages of trainable oontroUers are (i) 
the synthesis of the closed loop optimal controller is 
aooompUshed simultaneously with training, (ii) the 
controller, because of its special structure, has a higher 
reallability to oon^onent failure than'other more conventional 
designs and (ill) the trained controller provides a complete 

definition of the closed loop control law. from only a small 
training set, ’ 

We have alreadj seen that the choice of code affects 
the generalization characteristics and the conTergence late 
of learning process to a large extent. It was also pointed 
out that the controller using a contrast type code is more- 
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These experiences indicate that it is not necessaiy 
to impose too stringent design requirements on the performance 
of the components of a learning machine, i.e. a greater 
tolerance value is permissible for operating magnitude and 
reliability of a component can be low. This specially makes 
the use of such machines more attractive in a hazardous and 
hostilf environment. 

We have also suggested a method for realizing fuel 
optimal control law by using a network of two Adelines in 
parallel. It is easy to demonstrate that for n— level of 
input control a minimum of n-1 Adalines is needed, along 
with an (n-1) bit decoder. In the previous chapter in case 
of fuel optimal learning control systems a particular 
control level (+1, 0 or -1) was observed to be identified 
if there exists a plane which separates the members of that 


class from all other classes. This suggests that for n- 
v^ontrol levels, a minimum of (n— 1 ) Adalines in parallel 
would be needed. It will be noticed that in method suggested 
by Mendel , the training of the various Adalines tends 
to be independent of each other, and this usually leads to 
hslow convergence of the training process (This also has V 
been pointed out by Filsson[26]) . The alternative method ., 
suggested ^ Bj- the ' use 'of , 

r,. 
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independent, and it has been found that this method leads to 
a faster conTergenoe, 

The learning machines which we studied here were 
linear machines . An instance may occur where a linear machine 
fails to classify all the training patterns correctly (i.e. 
a nonpro^ectable dichotomy). Iii such a case it is desirable 
to use a cascaded netwoife of several Adalines. l’ig.45;.1,i 
shows a typical network of Adalines and the form of dichotou^y 
realized by this network. A system of such Adalines is 
called a layered machine in which the outputs of a layer of 
Adalines constitute input stimulus to another layer of Adalines 
and so on. A layered machine provides flexibility of 
processing the input stimulus before it is finally classified. 
Unfortunately little is known about the full potential about 
these machines (i.e. ^efficient training algorithms are r- 1 
known and no formulae yet exist to calculate their capacity). 
Detailed discussion in this field can be found in Rosenblatt 
[51], Uilsson Cadzow [ 3 J] and Holdermann £33]. 

and Block f^^J* ■' . 
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